Fix reject-inference.md Script Execution Output to match its own code - #567
Merged
yakew7 merged 1 commit intoSep 10, 2026
Merged
Conversation
…closes yakew7#522) The 'Script Execution Output' block and the analysis after it were pasted from an earlier version of the detection code and never regenerated. The extracted python block, run directly (seeded, byte-identical across runs), produces near-zero age gaps and ~0.92-0.95 population AUC across all three models - not the 15.80 -> 2.30 point gap and 0.7812 -> 0.8345 AUC the doc claimed. That is not just a stale number: it is structurally unreachable with this simulation. y_true carries no age term and the models train on credit_score and income_k only, both drawn independently of age_young, so selection acting on age alone is ignorable for estimating P(Y|X) and a RandomForest recovers a near-parity score distribution with or without IPW - there is no naive-model gap for the correction to close. Replaced the output block with the real deterministic figures and rewrote the analysis to describe what the code actually demonstrates: the disparity lives entirely in the selection gate (older approval ~50%, younger ~35%), invisible to any audit run on model scores or on the approved-only rows - which is exactly 'Why It Matters' item 2. Added a note that the low-order digits of a seeded RF can shift across CPU/BLAS backends. Regenerated the HTML, package mirror, and llms-full.txt section.
propcgamer20-png
requested review from
Shreyash0712 and
yakew7
as code owners
September 10, 2026 09:44
Contributor
|
@propcgamer20-png is attempting to deploy a commit to the yashkewlani2020-gmailcom's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
@yakew7 @Shreyash0712 - new PR touching a path you own, please review. |
yakew7
added a commit
that referenced
this pull request
Sep 10, 2026
Documents the newest explainer (#565), the CODEOWNERS grant to @propcgamer20-png, 8 doc/code fixes (#557-564, #566, #567), and the two direct fixes made during this review: the README.md regression from #557's broad substring replacement, and the recurred llms.txt/ROADMAP.md explainer-count drift after #565. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
yakew7
added a commit
that referenced
this pull request
Sep 10, 2026
- Moved @propcgamer20-png into Core contributors (43 merged PRs, now code owner for /Makefile and /scripts/), with their bio extended to cover the #558-567 batch. - Added @slsgzs-cloud (3 merged PRs, first merged 2026-09-08), missing from this file despite #503/#504 already being merged. - Snapshot bumped to 2026-09-10, covering through PR #567. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #522.
The Script Execution Output block and the paragraph after it were pasted from an earlier version of the detection code and never regenerated. Running the file's own python block directly (seeded, byte-identical across repeated runs) gives:
not the claimed
0.7812 -> 0.8345AUC and15.80 -> 2.30point gap.Why regenerating the prose (not just the table) was necessary
The old narrative ("15.80-point gap corrected down to 2.30") is not reachable with this simulation, not merely stale:
y_trueis generated with no age term - older and younger applicants repay at the same ~55% rate.credit_scoreandincome_konly, both drawn independently ofage_young.So selection acting on age alone is ignorable for estimating
P(Y | X); a correctly specified RandomForest recovers a near-parity score distribution with or without IPW, and there is no naive-model gap for the correction to close. IPW/parceling only bite under MNAR selection or when the outcome model omits a feature that drives selection - neither of which this minimal simulation creates.Change
RandomForestClassifier's low-order digits can shift across CPU/BLAS backends (same cross-platform effect discussed in model-drift.md cites a fabricated single-run gap/p-value for unfair.py (6.39%/p=0.348 instead of the real 7.16%/p=0.2564) #521).reject-inference.html,faircode/_explainers/reject-inference.md, and thellms-full.txtsection.No code in the fenced block was changed.
check_em_dash.py,check_broken_links.py,check_generated_files_current.pyall pass.